Improving Image Distance Metric Learning by Embedding Semantic Relations

Learning a proper distance metric is crucial for many computer vision and image classification applications. Neighborhood Components Analysis (NCA) is an effective distance metric learning method which maximizes the kNN leave-out-one score on the training data by considering visual similarity between images. However, only using visual similarity to learn image distances could not satisfactorily cope with the diversity and complexity of a large number of real images with many concepts. To overcome this problem, integrating concrete semantic relations of images into the distance metric learning procedure can be a useful solution. This can more accurately model the image similarities and better reflect the perception of human in the classification system. In this paper, we propose Semantic NCA (SNCA), a novel approach which integrates semantic similarity into NCA, where neighborhood relations between images in the training dataset are measured by both visual characteristics and their concept relations. We evaluated several semantic similarity measures based on the WordNet tree. Experimental results show that the proposed approach improves the performance compared to the traditional distance metric learning methods.